Image and object retrieval has been an active research topic for decades due to its desired applications in, for example, web image search, mobile visual search and personal photo management. Many conventional retrieval techniques adopt the bag-of-words model. FIG. 1 illustrates a bag-of-words framework for large-scale image search given a collection of images 100 and a query image 102. In this model, a visual vocabulary 104 is built by clustering on a large collection of local features such as SIFT. In the retrieval stage, each extracted feature from the query image 102 is assigned to its closest visual word in the vocabulary 104. The query image 102 is accordingly represented by a global histogram 106 of such visual words, and matched with database images 100 according to tf-idf (term frequency-inverse document frequency) weighting.
A fundamental problem in object retrieval techniques using the bag-of-words model is its lack of spatial information. Various techniques have been proposed to incorporate spatial constraints into the bag-of-words model to improve the retrieval accuracy. However, these techniques tend to be too strict or only encode weak constraints so that they only partially solve the problem for limited cases. While the bag-of-words model works generally well benefiting from its effective feature presentation and indexing schemes with inverted files, it still suffers from problems including but not limited to, the loss of information (especially spatial information) when representing the images as histograms of quantized features, and the deficiency of feature discriminative power, either because of the degradation caused by feature quantization, or due to its intrinsic incapability to tolerate large variation of object appearance.
tf-idf
The tf-idf weight (term frequency-inverse document frequency) is a weight that may be used in information retrieval and text mining. This weight is a statistical measure used, for example, to evaluate how important a word is to a document in a collection or corpus. Variations of the tf-idf weighting scheme may, for example, be used by search engines as a central tool in scoring and ranking a document's relevance given a user query.